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(57) Abstract: System and method for improving the performance of marketing campaigns in which advertisements or other mes- 
sages are distributed over an interactive measurable medium such as the Internet (106). System for autonomously selecting attributes 
of a message to improve message performance in an interactive measurable medium. System includes a database storing message 
attribute configuration information, including future message attribute configuration information, and past message performance 
report information; an optimization engine (102) receiving the past performance information and making decisions based on the 
received past performance information and predetermined rules to generate future message attribute configuration information; and 
a message server proxy (112) that interacts with an external message delivery system over a communication link to configure the 
external system to deliver the future message based on the future message attribute configuration information. Method for improving 
the stage-to-stage performance of a message. Method includes: reading prior stage message which includes a cumulative number 
of trials and successes for a message from the prior stage; reading message performance results; computing a current message; and 
generating a current message allocation. Computer program and product are also provided. 
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SYSTEM, METHOD AND BUSINESS OPERATING MODEL OPTIMIZING THE PERFORMANCE OF ADVER- 
TISEMENTS OR MESSAGES IN INTERACTIVE MEASURABLE MEDIUMS 
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Related Applications 

U.S. Patent Application Serial No. / (Attorney Docket No. A- 

24 69258/RMA), filed 2000 and entitled Method, Algorithm, and Computer 

Program for Optimizing the Performance of Messages Including Advertisements in 

an Interactive Measurable Medium; U.S. Patent Application Serial No. / 
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and Method for Optimizing the Performance of Email and Other Message 
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30 entitled Optimizing the Performance of Emails, are related applications each of 

which is hereby incorporated by reference. 
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Field of the Tnvention 

This invention relates generally to systems, methods, and operating models 
for optimizing the performance of advertisements and other messages or content, 
and more particularly to systems, methods, and operating models for optimizing the 
performance of advertisements, messages, and other content in an interactive 
6 measurable medium such as a global network of interconnected information devices 
and appliances. An example of such a global network is the Internet. 

BACKGROUND 

This invention relates to executing and optimizing the performance of 
marketing campaigns in which advertisements or other messages are distributed over 
12 an interactive measurable medium such as the Internet. The marketing campaign 
involves a list of ad alternatives and a target customer population. The goal of the 
marketer is to allocate the ad alternatives to the customer population to optimize 
business objectives such as maximizing the number of responses received. 

Various systems currently exist for the delivery and tracking of 
18 advertisements on the internet, for instance, ad servers for serving and tracking 

"banner ads" on a web page. The users of these ad delivery or ad server systems 
have access to data on the performance of all the ads on all the locations. This data is 
updated by the delivery and tracking system on a periodic basis. The users are also 
provided with an array of parameters to configure the delivery and tracking system. 
In a typical conventional situation, an advertiser buys advertising space (ad space) 
24 on a number of web sites. The advertising space buy on each web site consists of a 
number of categories. Such categories may correspond to different sections within 
that web site, where a section is a specific web page or a set of related web pages 
within the site. A category may also correspond to keywords searched by a customer 
on a search engine. The term "zone" will be used to represent a unique site and 
category combination. There may typically be a number of banners that an advertiser 
30 wishes to deploy across these zones. A banner is either a graphic image that 
announces the name or identity of a site, a text link, or a rich media file. An 
impression occurs when an Internet visitor sees a banner. A clickthrough occurs 
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when a visitor to a zone clicks on a banner. This redirects the visitor to the page on 
the advertisers web site. 

We will use the term "placement" to refer to a particular banner-zone 
combination. The fraction of impressions (relative to the total number of 
6 impressions served for the particular zone) that should be allocated to a particular 

banner alternative is an important placement parameter that the advertiser can select 
and modify, to boost the advertising campaign performance. 

Impressions can occur at any time, for example, whenever someone visits the 
appropriate page of a web site. However, the reports are typically updated at 
12 discrete times. We will call the intermediate time between two reports as a stage. At 
the end of each stage, the results are available for that stage. In particular, the 
following information is available for each placement: (1) the number of 
impressions delivered during a stage, and (2) the number of clickthroughs generated 
during a stage. 

1 8 Additionally this information (that is, the number of impressions delivered 

during a stage, and number of clickthroughs generated during a stage) may be 
available separately for: (a) different time slots within a stage (e.g. hour of day, if 
each stage is a day); (b) different frequency levels i.e. the number of times that an ad 
was seen by the customer; (c) different operating systems used by the machine on 
which the customer is logged on; (d) different internet browsers used by the 

24 customer; and(e) different domain addresses of the computer on which the customer 

is logged on. This list is exemplary and not intended to be exhaustive. 

In the current state-of- the-art, these reports are provided in printed form or 
in the electronic equivalent of printed form (for example, web-pages, spreadsheets, 
charts, or the like), and are manually analyzed by trained analysis personnel to 
30 derive new, improved advertisement configurations. For example, they are analyzed 
in an attempt to improve or optimize the clickthroughs generated by a pool of banner 
alternatives for a given zone, a given frequency level, and the like configuration 
information. This manual process is tedious and error-prone and has an inherent 
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delay between the period of data collection and the time new advertisements are to 
be placed because of the large amount of data to be analyzed and a large number of 
parameters to be modified. Even if errors are not made and the user is able to 
overcome the tedium of the process, it is unlikely to yield optimal or even near- 
optimal recommendations for advertisement configurations. This is especially true 
6 in light of the typical delay of from a day to a week that lapse between data 
collection, analysis, and a new or modified ad campaign based on the analysis. 

Optimization to provide an effective advertising campaign is in essence a 
multi-dimensional optimization problem, that by-and-large cannot be timely solved 
using conventional tools, methods, or systems. Furthermore, there are optimizations 

12 on multiple dimensions that are impractical to do manually and that will exacerbate 
the time delay. It is noted that these problems exist substantially independent of the 
type of advertisement or message, and that such issues and problems exist relative to 
advertisements for products and services, political campaigns, ballot measures and 
initiatives, media programming, lobbying, surveys, polling, news headlines, sports 
scores, as well as other directed marketing, promotions, surveys, news, information, 

1 8 other content generally, and the like. 

Therefore, there remains a need for an automated system for optimizing 
allocation parameters for advertisement alternatives or message alternatives. There 
also remains a need for an automated system and method for rapidly and efficiently 
executing the optimized allocation parameters to place the advertisement or message 
24 on the Internet or other local or global communication system. 

There also remains a need for a system and method that can learn and 
optimize across the various other parameters that can be reconfigured in 
advertisement delivery systems also commonly referred to as ad servers. There also 
remains a need for an ad server system and method that permits and advertiser to 
30 display different banners (or other content or messages) based on a time-of-day user 
web browsing profile which may include geographic location information, 
demographic information, or the like, as well as other user targeting information. 
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Each banner can be described in terms of a set of attributes. An "attribute" is 
an important, characteristic of an advertisement alternative. For instance, in a given 
situation, a banner ad may be conceptualized as consisting of three attributes: banner 
background color, banner message, and banner picture. There may be several 
choices available for each attribute. We will use the term "level" to represent 
6 different choices along the same attribute. For example, there may be sixteen 

different banner background colors, ten different banner messages, and twenty 
different banner pictures from which an advertiser might ideally choose or select. 
The value or level along each attribute may have a profound and significant impact 
on a users response to the ad. For example, in an advertisement for a pet store, 
selection of a banner picture of a cat rather than of a dog may significantly effect the 
12 success of the ad. 

Conventional ad server systems and methods treat individual ads as atomic 
units wherein constituent elements or parts, such as background color, message, 
picture, and the like are not separable. Hence, such elemental ad content attribute 
information is not available in conventional ad server systems and methods, which 

1 8 deal with the ad as an atomic unit. Therefore, there remains a need for system and 
method that generate a plurality of ads each having different elemental ad content 
attributes and which collects information on the success of such ads so that the 
system and method can learn which attributes are more effective in increasing 
performance of the message or advertising campaign, and then attempt to optimize 
the advertising campaign performance by generating an optimally placing a more 

24 effective pool of ads for subsequent campaigns. 

There also remains a need for a system and method that can learn so as to 
optimize the objective specified by the advertiser. For example, one simple but 
frequently used optimization objective is to maximize the number of user 
clickthroughs from a banner or other message to the advertisers home-page or other 
30 site. Another objective may be to maximize the clickthroughs such that the total 
sales generated by all the consumers who visit the store after clicking on one of the 
banner alternatives is greater than a predetermined amount. 
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There remains a further need to identify better performing or outperforming 
zones as quickly as possible so that the use and cost associated with poorer 
performing zones may be terminated in favor of better performing zones. 

BRIEF DESCRIPTION OF THE DRAWINGS 
6 FIG. 1 is a diagrammatic illustration showing a first embodiment of an 

overall system and operating model according to the invention. 

FIG. 2 is a diagrammatic illustration showing a second embodiment of an 
overall system and operating model according to the invention. 

FIG. 3 is a diagrammatic illustration showing another detailed embodiment 
of the inventive system. 

12 FIG. 4 is a diagrammatic illustration showing an exemplary embodiment of 

an Optimization Engine according to the invention. 

FIG. 5 is a diagrammatic illustration showing a flow chart of an exemplary 
optimization procedure according to the invention. 

FIG. 6 is a diagrammatic illustration showing another alternative 
embodiment of the inventive system. 
18 FIG. 7 is a diagrammatic illustration showing a different alternative 

embodiment of the inventive system. 

SUMMARY 

This invention relates to executing and improving and desirably optimizing 
the performance of marketing campaigns in which advertisements or other messages 

24 are distributed over an interactive measurable medium such as the Internet. A 
message (marketing) campaign involves a list of message alternatives (ad 
alternatives) and a target customer population. The marketer's goal is to allocate the 
message alternatives to the customer population to optimize business objectives such 
as maximizing the number of positive responses received. The inventive system and 
method provides an automates message targeting and allocation scheme that is 

30 dynamic and adaptive 

In one aspect the invention provides a system for autonomously selecting 
attributes of a message to improve message performance in an interactive 
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measurable medium. The system includes means for receiving message attribute 
configuration information and message performance information, an Optimization 
Engine receiving the message performance information and making decisions based 
on the received message performance information and predetermined rules to 
generate future message attribute configuration information, and a message server to 
6 deliver messages based on the future message attribute configuration information. 

In another aspect, the invention provides a system for autonomously 
selecting attributes of a message to improve message performance in an interactive 
measurable medium. The medium may be the internet. The system includes: a 
database storing message attribute configuration information, including future 

12 message attribute configuration information, and past message performance report 
information; an Optimization Engine receiving the past message performance 
information and making decisions based on the received past message performance 
information and predetermined rules to generate future message attribute 
configuration information; and a message server proxy that interacts with an external 
message delivery system over a communication link to configure the external system 

18 to deliver the future message based on the future message attribute configuration 

information. In one embodiment, the predetermined rules include past message 
attribute configuration information associated with the past message performance 
information. These predetermined rules may optionally further include message 
campaign data and optimization preference data. 

24 In another aspect the invention provides a method for improving the stage-to- 

stage performance of a message in a multi-stage message campaign in an interactive 
measurable medium, where the method includes the steps of: reading prior stage 
message state pertaining to a prior stage in a message campaign; the prior stage 
message state including a cumulative number of trials and a cumulative number of 
successes for a particular message at the end of the prior stage; reading message 

30 performance results representing message trials and message successes from the 
previous stage based on the prior stage state; computing a current message state on 
the basis of the prior stage message state and the message performance results; and 
generating a current message allocation based on the current message state. 
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Additional procedures and subsystems are provided that may be used 
separately and/or with external processing entities, or combined and integrated in 
various ways to yield different system configurations and processing distributions 
yet encompass the inventive concept. 

6 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 

This invention provides system, method, computer program, and operating 
model for optimizing the performance of advertisements and other messages or 
content, and optionally for placing, or serving the advertisement or other message or 
content to a target device and ultimately to the user or operator of that target device. 
12 Among other features, the present invention provides an automated system for 
optimizing allocation parameters for each message placement and executes such 
placements on the Internet. 

The invention can also learn and optimize across the various other 
parameters that are available for configuration message or ad server systems. An 

18 optimization algorithm is implemented as a computer program in an Optimization 

Engine to accomplish this. For instance, ad server systems permit display of 
different banner ads based on the time of the day the profile of the user browsing the 
web site (e.g. their geographic location and other demographic information), as well 
as other targeting criteria. The inventive optimizing system and method learn and 
then set the appropriate parameters across multiple content and placement 

24 dimensions so as to optimize the objective specified by the message provider, such 
as an advertiser. 

In addition, the inventive system and method gather user input and optimize 
for higher-level attributes that are more relevant to the advertiser. For instance, in a 
given situation, a banner ad may be viewed as consisting of three attributes: 
30 background color, message and picture. There may be several levels available to the 
user along each attribute. For instance, there may be five different background 
colors, four different messages, and five different pictures to choose from. The total 
number of banner ad alternatives which can be generated is equal to one hundred. 
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The information about these attributes has not heretofore been available in the 
traditional ad server systems, which deal with the ad as an atomic unit. By allowing 
the user to give attribute information for the different banners, the optimizing ad 
server system can learn which attributes and which levels within an attribute are 
more effective in increasing performance of the advertising campaign. This 
6 information is also extremely useful in generating a more effective pool of banners 
for the next campaign and determining optimal placements. 

Better performing or out-performing ads and zones for ads may also be more 
readily identified so that the use and cost associated with placing ads in poorer 
performing zones may be terminated in favor of placing such ads in better 

12 performing zones. This can be done, for example, by first estimating the 
clickthrough rate on every zone (or clickthrough rate of the best performing banners 
on the zone) followed by choosing the zones which generate the highest clicks per 
unit cost. Once the poor performing zones have been found, the advertiser would 
like to stop showing impressions on them. However, many of these ad zone buys are 
long-term (for example, from a month to two or more months) in nature and 

1 8 contractual obligations may not allow the advertiser to stop using the zone and stop 

paying for the impressions already bought. Even in the situation where a few weeks 
lead time is required to stop using a buy zone, the advertiser may swap the 
impressions bought with another advertiser such that the overall performance may 
improve for both the advertisers. The provision of the inventive method may 
therefore provide a method for interchanging ad space among advertisers or other 

24 businesses. These exchanges may be facilitated by a system and method that is 
automated and provides rapid analysis of performance data and rapid allocation of 
ads among the full pool of available ad allocation options. 

Those workers having ordinary skill in the art will appreciate, in light of the 
description provided above and in greater detail below, that the system as described 
30 above can be easily extended to be useful for optimizing message types other than 
internet banner ads, including, for example: (1) other forms of advertisements such 
as emails, (2) message alternatives for marketing channels like pop-up windows, (3) 
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web content alternatives for a web page, and (4) web layout alternatives for web 
page layout. 

In one aspect, the invention is a system for the optimizing ads in an 
interactive medium and automating the configuration of an ad server system in a 
6 larger interactive medium, such as the internet. The system includes an optimizing 
engine and an optimization algorithm or procedure that takes as input an ad 
performance report and generates recommended message or ad configuration 
parameters for the optimal placement and targeting of ads. It loads the reports from 
the database and stores the parameter values into a database. The database stores 
user-configured information, Optimization Engine generated configuration 
12 information, and the performance report information from the ad server system. An 
advertising server interaction service is operative to loads performance reports from 
the ad server system and store them in the database as well as taking user or 
Optimization Engine generated settings and setting them into the ad server system. 
Finally, a user interface allows the user to enter campaign data and specifications 
about the optimization. 

18 

In another aspect, the invention provides a business or operating model for 
providing advertisement, message, or other content optimization and placement. 
The operating model is applicable for business-to-business electronic commerce as 
well as business-to-consumer and consumer-to-consumer electronic commerce (e- 
commerce). 

24 

Embodiments of each of the major elements of the system that contribute to 
the operation of the system, as well as certain optional but desirable elements are 
now described. 

With reference to FIG. 1, there is illustrated a simple embodiment of the 
30 inventive system in the context of an internet advertising optimization environment, 
but the system and method readily applies to other types of messages on other 
measurable communication medium. Ad Optimization Service 102 receives data or 
information over the Internet 104 concerning ad alternatives and the performance 
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results of these alternatives in trials of campaign stages so far completed. 
Procedures that utilize ad states and ad performance as well as any constraints on ad 
placement are used by optimization procedures within the Optimization Engine and 
service. 

The advertiser 106 or other message manager is a point of control and may 
receive reports as to any unusual conditions or performance results. The advertiser 
106 is also the most likely entity pay fees for the ad optimization service and to reap 
the rewards of the improved ad performance. Typically, however, the advertiser is 
not involved in stage-to-stage allocation decisions as the process is completely 
automated and does not require human analysis of decision making. 

The ad target device 108 and the user of that device, typically a consumer 
with a personal computer and a web browser such as Microsoft Internet Explorer, 
receives one or more targeted ads (or other messages) and either ignores them 
(unsuccessful trial or impression) or positively responds to it (successful trial or 
impression) such as by clicking through a banner ad to the linked web site 
corresponding to the banner ad. 

Ad content publisher 110 provides the content ad alternatives for which 
optimization of allocation is desired. A single ad content publisher may provide all 
of the alternatives or a plurality of different ad content publishers may provide the 
different alternatives. 

The primary interaction of the ad optimization service 102 is with the ad 
server 112. The ad server 112 is responsible for actually placing the ads and for 
collecting performance results, such as success rate or clickthrough rate, for the ads 
so placed. It is from the ad server service 1 12 that ad optimization service receives 
performance information and to which ad optimization service sends ad placement 
allocation directives or recommendations. 

In some embodiments, the ad optimization service 102 and the ad server 112 
of the FIG. 1 embodiment may be combined into a single ad server and optimization 
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service 1 14 as illustrated in FIG. 2. The ad content publisher may also be combined 
with the ad server and/or with the ad server and the ad optimization service. Other 
combinations are also possible and not enumerated here. 

This description applies, with changes that will be clear to workers having 
6 ordinary skill in the art, to messages other than advertisements. For example, the ad 
optimization service becomes the message optimization service, the ad server 
becomes the message server, and so forth. 

The invention therefore also provides a business model and method or 
operating model. In one embodiment of the inventive business model and method of 

12 doing business, referred to as the web hosted model, the User-Interface and Message 
Optimization are on a common web server site. The Optimization system and 
service interact with the underlying ad server system and a user of the system is 
levied a fee for each message trial or in the context of banner ads, for each 
impression. Volume discounts for different numbers of impressions or levels of use 
may advantageously be implemented. Revenues may also be collected on a per 

18 campaign basis for setting up the campaign and for analyzing the performance 
results and recommending allocations. Additional fixed or variable fees may be 
levied on a per- week or per-month basis. 

In an alternative business model, the optimization system and method are 
provided to message agencies, such as for example, to ad agencies so that the 
24 Optimization Engine and service are integrated into a larger message or ad service 
system. Revenues are generated on the basis of the number of licensed installations 
and may be fixed, volume dependent, or charged as a percentage of revenues 
generated. Additional costs may be charged based on the size of system in terms of 
the total number of impressions delivered, size of the campaigns, size of the 
database, and the like factors. 

30 

Other aspects of the inventive business model may readily be appreciated 
from the description of the relationships between the Optimization Engine Service, 
the Database, the Message Server Interaction Service (MSIS), the Message Server, 
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and the User or Message Control Agent. These relationships among these entities 
and the information and/or services interchanged between and among them may 
readily be accomplished on the Internet or other interconnected network of 
computers or information appliances. The high-speed interchange and measurable 
characteristics of the Internet as a communication medium provide technical 
6 advantages and effects that could not have been earlier achieved. One particularly 
useful benefit of the inventive system, and business operating model lie in its ability 
to rapidly measure the success of multiple messaging (e.g. advertising) schemes and 
select those scheme alternatives that outperform other schemes. Advantageously, 
the system and method permit poorer-performing schemes to be controllably 
retained (or discarded) so that in the event performance changes, they are 
12 automatically given a greater proportion of the message trials or impressions. 

Reallocation of message alternatives may be made hourly, daily, weekly, or at any 
other time as the system is automated and at least in some embodiments directly 
collects performance information and allocates messages among message 
alternatives directly over the internet. 

18 Another embodiment of the invention is now described relative to FIG. 3, 

which shows additional detail of the User Interface, database, Optimization Engine, 
message server interaction service, internet, and message server. 

The User Interface (UI) 302 system gathers configuration information from 
the message sponsor, such as from the advertiser 106. The User Interface 302 may 
24 typically be a personal computer or other information device or information 

appliance that permits a user to enter data into the system and obtain reports to see 
what happened. In one embodiment of the system the user interface is a web site. 

Generically, this configuration information includes where the message (for 
example the ad or email) should be displayed or otherwise presented, the different 
30 creative media that is to be displayed, information about the different attributes of 
the creative media, parameters about different options for the placement and 
instructions to the Optimization Engine 306 and optimization algorithm executing 
within the optimizing engine about which objectives to optimize for and other 
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parameters, like "grace period", the period of time that the optimizing engine does 
not take action on the new creatives after they have been introduced. The grace 
period is an optional mechanism for the marketer to fine tune the optimization of 
his/her campaign. In an embodiment of the invention particular directed toward 
banner advertising on the internet, this configuration information includes a list of a 
6 banners, a list of sites and zones that the banners can be placed on, the different 
attributes of the banners such as background color, picture, message and animation, 
any specific placement constraints specified by the advertiser or ad agency attribute 
descriptions, and other optimization specifications and/or parameters. As with 
Message Servers (e.g. Ad Servers) 304, it is assumed that the message sponsor (e.g. 
advertiser or marketer) has already bought or otherwise contracted for impressions 
12 on these zones. 

When the User Interface 302 system has gathered the above information, it 
schedules an Optimization Engine 306 targeted job in job scheduler 316 job table 
317 of database 308 directing the engine service 306 to determine the initial set of 
allocations. This set of allocations is referred to as initial set of allocations because 

18 when new messages are placed into the system it needs to be determined where 
these are placed, and what fractions of the impressions for a zone that are going to 
be assigned to that message. The Optimization Engine service executing within 
Optimization Engine 306, on completion of the OE job writes an initial set of 
allocations for different placements into the database and schedules a Message 
Server Interaction Service 320 (e.g. an Ad Server Interaction Service) targeted job 

24 (MSIS Job) to modify the appropriate settings on the Message Server 304. 

Periodically or according to other rules or procedures, the Message Server 
Interaction Service (MSIS) 320 pulls new performance report information from the 
Message Server 304. These performance reports contain data on performance 
metrics for the messages. In the context where the message is a banner ad on an 
30 internet web site, the performance metric may be, for example, the total number of 
clickthroughs for a given banner ad at a given zone and the total number of 
impressions delivered for that ad. The Message Server Interaction Service 320 stores 
this performance information into the Database 308 and schedules an Optimization 
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Engine targeted job tasking the Optimization Engine service to generate a new set of 
message allocations. 

The Optimization Engine 306 and the optimization algorithm 307 
implemented by that engine provides the core piece of computer program code that 
6 implements the mathematical models to implement the optimization decisions. The 
engine service 309 wraps this Optimization Engine 306 to add the interaction with 
the database 308 and other parts of the system 300. One might conveniently use the 
two terms (Optimization Engine and optimization service) interchangeably but 
sometimes it is convenient to identify them separately. 

12 All the historical message allocations that have been made and the 

performance results that have been collected to date for a particular message 
campaign are stored in the database. Based on the performance report information 
available in the database, the Optimization Engine 306 models the performance of 
different messages and message attribute alternatives. The Optimization Engine 306 
uses this historical information to generate new message allocations from among the 

18 set of possible current message allocations that bias message impressions away from 

poor performing messages or message formats and/or attribute alternatives and 
toward what are or appear to be better performing messages or message formats. In 
the internet banner ad context, alternatives may include modifying the allocations 
for banner content (e.g. banner background color, banner message, and banner 
picture as applicable), zone (e.g. site-category combination), placement (e.g. banner- 

24 placement combination), different targeting attributes (such as for example, different 

banners for time of day, different geographical locations, browser types, frequency 
that ad has been shown, and any other demographic information that may be 
available about the end-user, their employer, or their operating environment), and 
the like message configuration information. 

30 One embodiment of the invention will also generate these banner ads (and 

other messages) automatically based on these various attribute-alternatives and 
deploy the banner ads (or other messages) that it learns to be most effective. Thus, 
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the inventive system provides for automated autonomous message optimization, 
generation, and placement. 



The system and method of the present invention can also optionally provide 
reporting and alert functions to inform the message sponsor such as the advertiser of 
6 significant events that might require their attention. This is desirable so that there is 
some human review of anomalous situations. Examples of some of the events or 
situations for which a special report or alert might be issued include, for example: (i) 
no impressions were delivered even though purchased by the advertiser; (ii) a large 
change in the type or number of impressions delivered; (iii) a large change in click- 
through in a given zone; (iv) a large change in a specific banner's performance; and 
12 (v) the click-through performance achieved is below a specified contract minimum. 
Other situations for which a report or alert may be generated may be specified for 
the system. Advantageously, the situations for which special reports or alerts are 
generated may readily be altered in real or near-real time, such as by interactive 
programming. The system and method also desirably provide the ability to monitor 
ones messages using the User Interface and to create custom reports. 

18 

The inventive system and method are inherently fault tolerant and can deal 
with a variety of potential failures including unavailability of the Message or Ad 
Server 304, and the unavailability of the most recent performance reports. In a 
situation where the message server 304 is unavailable, the system user (e.g. the 
message-sponsor) may still access past performance report information and current 

24 message allocations from the database 308 using the User Interface 302 instead of 
depending on the reliability of the underlying execution platform (e.g. the ad server). 
When the most recent or current reports are not available for what ever reason, the 
Optimization Engine 306 service can produce the best message allocations based on 
the information available in the historical data, and the time elapsed since the last 
report was obtained, even if all the reports, such as current performance reports 

30 which were due from the Message Server 304, but have not been received, are not 
available. 
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A scheduling infrastructure is provided for scheduling tasks or jobs in the 
system, particularly for scheduling jobs to be performed by the Optimization Engine 
306 and the Message Server Information Service 320. 

Part of this scheduling infrastructure is provided by a Job Table data 
6 structure 316 defined in the database 308 that is used for scheduling the various jobs 
in the system. Jobs are scheduled for both the Optimization Engine and for the 
Messaging Server Interaction Service. In one embodiment, the data structure of the 
Job Table includes the following columns: Service Type, Category, Scheduled Time, 
Status, Start Time, and End Time. More, fewer, or different data structure columns 
or organization may be provided to accomplish the functionality provided by these 
12 scheduling information items. 

• " Service Type " specifies the target system module which is expected to 
execute this job, for example, the Optimization Engine Service (OE), or the Message 
Server Interaction Service (MSIS). 

1 8 "Category" specifies the type of job which the service needs to execute. For 

example, in one embodiment, five job type categories are supported: (i) 
Optimization (OP), for the optimization engine; (ii) Performance Analysis (PA), for 
the Optimization Engine; (iii) Download Campaign (DC), for the Ad Server 
Interaction Service; (iv) Get Report (GR), for the Ad Server Interaction Service; and 
(v) Set Allocation (SA), for the Ad Server Interaction Service. 

24 

"Status" specifies the current status of the job within a service. In one 
embodiment, there are four possible values of this field: (i) New (N), indicating that 
the job has not yet been opened by the service; (ii) Work-in-Progress (W), indicating 
that the job is currently in progress; (iii) Success (S), indicating that the job was 
completed successfully; and (iv) Failure (F), indicating that the job was not 
30 completed successfully, for example, due to an exception or a system error. 

The Job Table 317 also provides time information. Those workers having 
ordinary skill in the art will appreciate that data and time information may be 
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provided in a variety of ways and formats. For example, absolute or relative time 
may be used, and that some time parameters may be computed from other time 
parameters. Therefore, the particular time parameters describe relative to one 
embodiment are exemplary. 

6 The Job Table "Scheduled Time " field indicates the time at which the job is 

scheduled to be performed. This scheduled time may be manually entered, but is 
desirably determined by the system based on scheduling factors, such as for example 
the loading of the job target, the frequency of the jobs for a particular campaign, the 
expected time of availability of input data such as the availability of performance 
report data prior to an optimization, and the like factors. In one embodiment, 
12 Scheduled Time includes date information and time of day information and is 
encoded in conventional format. 

The Job Table " Start Time " field indicates the actual starting time of the job. 
It may differ from the scheduled time. For example, if the job target has not 
completed a previous job or jobs or the loading on the job target is too great, the 
18 start of the job may be delayed somewhat. The " End Time " field indicates the time 
when the job was completed (successfully or unsuccessfully). Both Start Time and 
End Time may be considered to be optional parameters, however, in embodiments in 
which a service charge is levied based at least in part on the job computation time, 
knowledge of the actual start and end times permits computation of the elapsed 
computation time and provides an objective bases for service fees. 

24 

While each of these scheduling information items are present in the data 
structure of a preferred embodiment of the Job Table, it should be understood that 
not all of these information items is required in an operational system. For example, 
knowledge of the actual Start Time and End Time may be useful for monitoring the 
performance of the inventive system; however, the gathering or availability of such 
30 information does not affect the message optimization provided by the inventive 
system and method. In analogous manner, though knowledge that a job failed is 
advantageous so that the job may be rescheduled or other compensatory or 
corrective action taken, indications that the job is new or in-progress, ultimately 
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have no effect on message optimization. Finally, though scheduling at 
predetermined times is desirable, the inventive system and method may be 
established in an environment where the jobs are merely scheduled sequentially or 
on a first-in-first-out basis. In this later instance, appropriate consideration and 
control should be exercised such that a first job generating results needed as inputs 
6 to a second job is scheduled and completed before the second job. 



Table I. Exemplary Job Table for Database. 



JOB TABLE 


Service Type 


Category 


Scheduled Time 


Status 


Start Time 


End Time 


OE 


PA 


14 April 2000 
23:50:00 


S 


14 April 2000 
23:50:00 


14 April 2000 
23:50:32 


OE 


OP 


14 April 2000 
23:59:00 


S 


14 April 2000 
23:59:00 


15 April 2000 
00:00:09 


MSIS 


SA 


15 April 2000 
01:15:00 


w 


15 April 2000 
01:15:03 




OE 


PA 


15 April 2000 
01:18:00 


w 


15 April 2000 
01:18:00 




MSIS 


GR 


15 April 2000 
20:10:00 


s 


15 April 2000 
20:10:04 


15 April 2000 
20:11:09 


OE 


OP 


15 April 2000 
21:0:00 


N 







The Engine Service of the Optimization Engine 306 polls the Job table 317 
12 of the Database 308 at regular intervals (or according to some predetermined rules or 

schedule) for the list of pending jobs to process. The optimization engine 306 and 
the optimization algorithm implemented by that engine is the core piece of 
algorithmic and computer program code that implements the mathematical models 
to provide the optimization recommendations and decisions. The engine service 
wraps this Optimization Engine to add the interaction with the database and other 
18 parts of the system. For each job in the list of pending jobs, the engine service 
invokes the appropriate process in a new thread. More particularly, the engine 
service polls the database to identify any jobs that are targeted to the Optimization 
Engine (OE) and the Engine Service provided thereby. Those jobs in the Job Table 
having Service_Type=OE are targeted to the Optimization Engine and the engine 
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service. (Those jobs in the Job Table having Servdce_Type=MSIS are targeted to 
the Message Server Interaction Service.) 

Note that in some embodiments of the invention, the Optimization Engine 
306 does not poll the database 308 and alternative mechanisms are provided to 
6 inform the Optimization Engine (or the message interactive service interface) that a 
job is ready for processing or execution. Therefore, either or both polling and non- 
polling implementations may be provided. 

In one embodiment, the Optimization Engine 306 processes two categories 
of jobs: (i) Performance Analysis jobs and (ii) Optimization jobs. Normally, at least 

12 one performance analysis report is generated by a performance analysis job before 
an optimization job for the same message campaign is executed; however, an 
initialization set of performance data may be loaded in the database at the start of a 
new campaign so that optimization may begin. The initialization set of performance 
data may be established manually, generated by other analysis, be adopted from 
related campaigns, or by any other means. In one embodiment, such initialization 

18 set of performance data is a neutral or unbiased data that allows all candidate 

message attributes an opportunity to perform. In another embodiment, this data set 
is biased in favor of certain message attributes. 

A performance analysis job uses the data on the current sets of allocations 
and placements, the historical performance of each of these and the most recent 

24 performance report of the current allocations and placements in play as inputs and 
generates a new set of message allocations and placements (or in particular 
embodiments for an advertising message, what banner ads should be turned on or off 
or allocated in which ratio and which placements should be added or deleted) as 
outputs. The message allocations and placements generated by the Optimization 
Engine 306 are stored in the Database 308. (In some embodiments message 

30 allocations and placements may be cached for later use in a memory storage device 
of the Optimization Engine, or directly affect the message server, without 
intermediate storage in the database.) 
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An optimization job retrieves the message performance report information 
(or in particular embodiments directed to an advertising message, the ad 
performance report information) stored by the Optimization Engine in the Database 
and uses it as an input to generates message configuration parameters for the optimal 
placement and targeting of messages (or ads) as outputs, and stores them in the 
6 Database. 

Recall that the term campaign refers to a set of media purchases over a 
period of time by a particular message sponsor, usually linked to a specific business 
objective, over which the creative messages need to be optimized It is not necessary 
for a campaign to have a time limit. For instance, the problem of optimizing the 
12 daily new as items at a news site is a continuous process every day, without a set 
termination time. For an advertisement (or ad), an ad campaign is a set of sites and 
sections within the site that a number of impressions have been purchased to be 
served over a period of time. Where the message is email, the email campaign is a 
list of email addresses and a time over which mail is to be sent to this set of 
addresses. 

18 

In the context of the invention, state refers to the encapsulation of relevant 
cumulative knowledge/information gathered over time since a given start time of the 
Campaign. Two types of state may be used by the Optimization Engine: global 
campaign state and individual cell state. Global campaign state refers to the 
encapsulation of those properties/aspects of the Campaign that are relevant at the 
24 Campaign, but not at the cell level. A cell is a representation of a specific placement 

of the Campaign. A placement in turn is the mapping of the message (or any web 
content) at a given site. A single message (or web content) can be mapped to 
multiple sites, and so can result in multiple cells. The Campaign is thus comprised of 
all the cells for the placements in the Campaign. Individual cell state refers to the 
encapsulation of the cumulative knowledge/information for that individual cell. 

30 

Upon invocation of the Engine Service, the Optimization Engine process 
executes one or more threads to accomplish processing for a given message 
campaign; and can kick off the processing of multiple campaigns at the same time. 
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Processing is campaign based because each Campaign is completely independent 
from the other. Note that typically, the optimization and performance reporting are 
iterative in nature and that the longer the campaign runs, the greater the learning and 
optimization that can be accomplished. It is also interesting, that messaging trends 
and performance will frequently change over time and the iterative nature does not 
6 imply that the message criteria will become static once the campaign has run for a 
long period of time. Quite the contrary, as message saturation or staleness may 
occur after even a short period of time; such staleness being identified in the 
performance reports, and resulting in recommended changes by the Optimization 
Engine. 

12 The thread-based nature of the Optimization Engine 306 process is now 

described. The Database 308 is the persistent store for the Campaign states as well 
as all information gathered on a regular basis in the form of performance reports. On 
arrival of a new report or reports, a job is scheduled for the optimization engine to 
process the new report or reports on a per Campaign basis. New reports can arrive 
for multiple Campaigns at the same time. The Optimization Engine 306 process 

18 polls the database 308 on a regular basis for jobs that are ready to be run. Once the 
Optimization Engine gets a list of jobs that are ready to be run, it spawns a new 
thread 326 for processing each job independently of the other jobs. 

From a top-level perspective and in simplified terms, processing involves 
reading the previous state of the Campaign (global state as well as cell state), 
24 processing the new results and updating the state of the Campaign, and finally 
generating the new recommendations. The new states and recommendations are also 
stored in the database. 

The Optimization Engine and the process executed within the Optimization 
Engine are synonymous. The Optimization Engine 306 process executes the 
30 following steps: It reads in the Campaign state from the database and stores it into 
main memory data structures. Campaign state in one of the embodiments consists of 
a state for each placement, and a timestamp of the last state update. The state for 
each placement may, for example, comprise the cumulative number of impressions 
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and the cumulative number of clickthroughs. The most current campaign state is 
stored in the database. 

The Engine Optimization 306 then queries the database 308 for all new 
performance reports for that campaign since the last time the Campaign State was 
6 updated. Campaign state includes message state or data, performance results state or 
data, and allocation state or data. For each such report it: (a) Reads the report from 
the database 308 into Optimization Engine main memory 307 data structures; and 
(b) Processes the report and updates the campaign state data structures. Using the 
new state, and applying suitable allocation algorithms, it generates new allocations 
for the Campaign and stores the allocations into new data structures in memory. 

12 

One embodiment for performing the allocation and performance 
optimization is described in co-pending United States Patent Application Serial No. 

/ , (Attorney Docket No. A-69258 filed _ May 2000, and entitled Method, 

Algorithm, and Computer Program for Optimizing the Performance of Messages 
Including Advertisements in an Interactive Measurable Medium, and is hereby 
18 incorporated by reference in its entirety. This optimization algorithm and procedure 
is also described below. It should however be understood that although 
embodiments of these particular algorithms and procedures may advantageously be 
used, the inventive system and method may alternatively utilize other processing 
routines, procedures, or algorithms in attempts to improve or optimize advertisement 
or message performance. 

24 

The process now begins a database transaction. The Engine Optimization 
306 process stores the new Campaign states (including message state) from the data 
structures in memory 307 to the database 308. It also stores the new Campaign 
allocations (including message allocations) from the memory data structures to the 
database. Once these operations are successful, the process commits the database 
30 transaction. 

This database commitment process prevents corruption of data in the 
database 308 (which is the only reliable data in the system) by the intelligent use of 
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database transactions. A database transaction is a database feature that provides the 
application the ability to make multiple changes to the database 308 in an atomic 
fashion, that is, with the guarantee that either all changes to the database happen or 
that no changes happen. This prevents the possibility that the data in the database is 
updated only partially. In this particular embodiment, no other process in the system 
6 updates the Campaign states or generates allocations. This ensures the consistency 
and reliability of the Campaign states and allocations in the system. 

Clearly, the above embodiment of the optimization procedure is just one 
possible embodiment of the algorithm that can be used for optimizing the total 
number of responses. Algorithms based on other techniques such as genetic 
12 algorithms, neural networks, machine learning, stochastic optimization or a 
combination of these techniques can be used instead of the algorithm described. The 
overall system architecture is independent of the technique used. 

It can therefore be seen from the above description that the inventive 
optimization algorithm and procedure as well as indeed overall inventive system and 
18 method provide particularly advantageous benefits in the banner ad messaging 
context. The methodology applied to banner ads can readily be extended to other 
messaging situations, for example to email, interactive television, and the like. 

In one embodiment, the core engine optimization procedure is implemented 
as computer program software procedures written in the Java language for maximum 
24 flexibility and also exposes a COM Application Program Interface (API). However, 
those workers having ordinary skill in the art in light of the description provided 
here will appreciate that various other language may alternatively or additionally be 
used. 

The Message Server Interaction Service (MSIS) manages the interaction 
30 between the message product (such as for example an banner ad product, an email 
product, a splash page product, a news page product, a portal home page product, or 
other message product) and a message server. In one embodiment, the message 
server is an external internet based message server. In other embodiments the 
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message server is integrated with the inventive system including the database, MSIS, 
and Optimization Engine. In either of these embodiments, the message served 
includes ads or advertisements and the message Server Interaction Service (MSIS) is 
referred to as the Ad Server Interaction Service (ASIS). The MSIS (ADIS) is 
responsible for retrieving performance reports from the Ad Server (or Ad Delivery 
6 Service) setting banner ad placements and allocations in the Ad Server, and 
downloading campaign settings from the Ad Server for use by the Optimization 
Engine and the User Interface. 

The ADS advantageously utilizes a plug-in architecture, where the interface 
to the message server (e.g. to the Internet Ad Server) is a replaceable module which 

12 exposes a common application program interface (COM API). A "plug-in" is a 
module that implements this interface for a particular provider (e.g. a particular Ad 
Server). In the case of an external internet based Message Server, the plug-in is 
called an Agent (but this distinction between plug-in and agent is primarily 
semantic), accesses the Internet Message Server (or Ad Server) through the HTTP 
protocol. (Where communication links other than the internet are utilized, alternative 

18 supported protocols may be used.) The Agent within the MSIS simulates the 
interaction of a Internet Message Server customer, requesting pages of data via 
HTTP and parsing the needed information out of the returned HTML data. Other 
Agents would get the data in XML, or other proprietary format, based on what the 
underlying message server supports using HTTP:, FTP:, HTTPS: or another public 
or proprietary protocol. The information extracted from the returned data includes 

24 ad properties and names, zone properties and names, buy properties and names, 

campaign properties and names, performance data (such as clicks and impressions). 

The parsing procedure includes the steps of deconstructing the HTML into 
its constituent tag structure and locating the desired data element at a known place 
within the tag structure. Standard parsing techniques of tokenizing the text stream 
30 and applying a simple grammar to parse out the HTML is used. Those workers 

having ordinary skill in the art in light of the description provided here will 
appreciate that any standard parsing technique could be used to accomplish the 
result. 
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The MSIS (ADIS) interfaces with the Optimization Engine and the User 
Interface via the database. The MSIS (ADIS) retrieves jobs targeted for the MSIS 
(recall Service_Type=MSIS in the Job Table identifies MSIS targeted jobs) from the 
database and acts on the job, placing the results (performance report information) 
6 back in the database 

The MSIS (ADIS) performs three types of jobs: Get Report, Set Allocations, 
and Download Campaign. For a Get Report (GetReport) job, the ADIS queries the 
Internet Ad Server for a detailed listing of impressions and clicks for each 
advertisement in the campaign. Recall that an impression is an instance of possible 

12 exposure of the message to the end-user, and clicks represent one possible 
performance indicia as to the successfulness of the impression. Other measures of 
success may alternatively be used - for example, whether the user was actually 
converted into a customer for the goods or services offered by the advertiser, what 
was the lifetime value of the customer to the advertiser or whether the brand of the 
advertiser was recalled more successfully after the exposure. The downloaded 

18 information is stored in a staging table in the databases, using for example, SQL 
server's BCP fast copy operations, and a stored procedure performs transformations 
on the data to put it in the correct databases tables from where it would be used by 
the other system components tables. The staging table allows the batch upload a set 
of data containing potentially invalid elements, and then the execute of a verification 
process against the existing data in the database. The other difference between the 

24 staging table and the report table is that the report table contains key constraints 
between it and other tables in the system where the staging table does not. This 
means that the staging table has the requirement that each row of data in it have a 
corresponding entry in some other table elsewhere in the database. The staging table 
cannot have this requirement, as we do not make the effort to insure that the related 
elements exists in the related tables before bulk-uploading the data. This is an 

30 optimization that allows fast upload of data into the staging tables. 

For a Set Allocations (SetAllocations) job, the ADIS reads the current 
allocation settings for a campaign from the database and for each advertisement with 
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a current allocation, the ADIS issues a request to the Internet Ad Server to modify 
the advertisement's properties in conformance with the allocation. Recall that an 
allocation specifies how the impressions are to be allocated amongst the different 
placements in the system. Typically, a Set Allocations job consists of a batch of 
allocations to be set for the entire campaign at the same time, rather than an 
6 allocation specific to one ad placement. Setting Allocations for more than one ad 
placement at the same time reduces the number of connections required with the 
external Message Server (Ad Server). The inventive system and method may 
alternatively be implemented without batched allocation setting. 

For a Download Campaign (DownloadCampaign) job, the ADIS queries the 
12 Internet Ad Server for a data listing buys, sites, zones, and advertisements for a 
given customer. Recall that a zone is a unique site and category combination, and a 
"buy" is a combination of the zones on a site at which the ads have been purchased. 
The resulting data from the Ad Server is stored in a local database within the system 
and is visible to the message product (e.g. banner ad product) customer through the 
User Interface pages. 

18 

Having described the system and method generally, we now focus the 
description on several exemplary applications and a particular exemplary 
optimization algorithm and procedure that may be used with the inventive system. 
These particular applications and embodiments are described for purposes of 
illustration and not by way of limitation. 

24 

Exemplary Embodiment of an Optimization Engine and Procedure 

As the inventive method is advantageously implemented as a computer 
program on a general purpose computer, such as for example a computer operating 
in an Internet or other network server environment, attention is now directed to an 
exemplary computer system which may be used in conjunction with the inventive 
30 method, procedure, computer program, and computer program product. 

With respect to FIG. 4, computer 102 includes a processor or CPU 104 
coupled to a fast random access memory 106 storing data 108 and procedures 110. 
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Processor 104 is also conveniently coupled to one or more mass storage device 112 
such as one or more hard disk drives of which many types are known, The mass 
storage devices may typically store information in one or more databases. In the 
context of the present invention, such databases may store performance data, 
allocation data, and other data and/or procedures or algorithms that are not currently 
6 being processed by the processor 104 or memory 106. Input/output devices 114 

may also be conveniently coupled to the processor or other computer system 102 
components for inputting data to the computer, retrieving results from the computer, 
monitoring activity on the computer or on other network devices coupled to the 
computer, or other operations as are known in the art. A network interface 116, such 
as a network interface card (NIC), modem, or the like is also conveniently provided. 

12 

Procedures 110 may for example, include an operating system 120, an 
optimization procedure 122, a state vector update procedure 124, a message 
allocation procedure 126, a pair-wise or other message alternative comparison 
procedure 128, as well as other procedures and algorithms 130. 

18 Data 108 may for example include one or more state vector 142, results 

vector 144, allocations vector 146, constraints list or constraints items 148, stage 
discount factors or parameters 150, threshold values 152, proportional of 
impressions or trials factors 154, or other data or parameters 156. 

Those workers having ordinary skill in the art in light of the description 
24 provided here will appreciate that the computer system 102 described here is merely 
exemplary and that various modifications may be made, or that other computer 
system architectures may be used in conjunction with the inventive system, method, 
and computer program. 

Optimization Procedure and Algorithm 

30 Having described certain high level features of the inventive optimization 

procedure as well as providing an exemplary environment in which the inventive 
procedure has particular applicability, attention is now directed to details of a 
particular embodiment of the inventive algorithm or procedure. 
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This embodiment is described in the internet banner ad context. In this 
application context, three principal types of data are used by the inventive 
procedure: (i) State Vector, (ii) Results Vector, and (iii) Allocations Vector which 
includes a Constraints List. (In one embodiment, the Constraints List is separate 
6 from the Allocation Vector, but used to generate the Allocation Vector.) These data 
types are described in turn. The term vector is used to refer to one form in which the 
data is stored and/or manipulated. 

Zone/banner States and State Vector. For each combination of a zone and a 
banner, the inventive procedure retains a "state vector" that records all of the 
12 information that has been collected on the given banner at the given zone 
(zone/banner combination). This state vector is updated at the end of each stage of 
the campaign to reflect the results of that stage. In particular, the state vector 
contains the history of impressions and clickthroughs for the given zone and banner. 

Results Vector. The results vector is a vector of the results of the most 
18 recent stage of the campaign. It contains the unprocessed impression and 
clickthrough data for each zone/banner combination. If a banner was not displayed 
at some zone during the most recent stage, the results vector indicates this fact. 

Allocations Vector. The allocations vector contains the recommended 
allocation of banners to impressions for the next stage of the campaign. Intuitively, 
24 these allocations are expressed in percentage terms. However, in practice the 
recommended allocations may be re-expressed in a form acceptable to the ad server. 
Hence, some translation or transformation may be provided. The Allocations Vector 
may also include a list of constraints, though the constraints list may be separately 
provided. 

30 A list of any constraints on the allocation of banners to impressions is stored 

in the constraints list. At least two types of constraints can occur. First, the user can 
specify the minimum number of banners that must be retained and displayed at each 
stage of the ad campaign. Second, the inventive procedure implemented as a 
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computer program on a general-purpose computer retains a table of "banner 
exclusions." The table of exclusions, includes for example, a table defined as a data 
structure in memory that indicates when a given banner should not be shown on a 
given zone. This table may include one or more entries or may be empty. 

6 The inventive procedure is advantageously run at the end of each stage of an 

ad campaign. It allocates the available banner impressions or banner ad alternatives 
at each zone for the next stage among the available banners. That is, it generates a 
strategy or recommendation for displaying banners during the next stage of the 
campaign. In developing this recommendation, the algorithm uses the results from 
all prior stages. (Though in at least one embodiment, a selected range of data from 

12 older stages may optionally be discarded, and in a different embodiment, not all data 

from all stages has the same weight or value in the recommendation.) One objective 
is to allocate the banners at all stages so that the total number of positive results, 
such as user clickthroughs on a web site, recorded at all zones by all banners during 
the campaign is maximized. Other operations may optionally be performed, for 
example, during the ad campaign, the client advertiser or marketing manager may 

1 8 wish to delete or add a zone or banner. 

An embodiment of the optimization procedure 202 is now described relative 
to the flow-chart diagram of FIG. 5. First, the prior stage message state is read (Step 
204). Next, message performance results are read (Step 206). These performance 
data or results are typically received from another source, such as an ad service and 

24 stored (Step 208). The current (or new) message state is then computed (Step 210) 
using, for example, the state vector update procedure described in greater detail 
elsewhere herein. Message allocations, possibly with some format transformation to 
adapt them to the particular format needs of a receiver, are generated (Step 212) and 
sent to the receiver such as a message service (Step 214). As a primary application 
of the inventive method is for Internet advertising, the message service will 

30 frequently be an ad service. As the message campaign (ad campaign) is generally a 
multi-stage campaign, the procedure is repeated until all stages in the campaign have 
been completed. This requires that the prior message state used in step 204 be 
replaced with the current message stage (Step 216) generated in step 210 before the 
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next iteration (Step 218). New message performance results received in step 206 are 
also used for the next iteration. 

While it is generally intended that performance data or results are received 
for each prior stage before allocations are generated for the current or next stage in 
6 the campaign, and the inventive procedure benefits from such timely receipt, the 
invention does not require such timely receipt. In the event that expected prior stage 
results are not for some reason received as expected, the inventive procedures are 
sufficiently robust that the new allocations are merely based on the previous 
performance data or results. Therefore, when new performance results are not 
available, the old performance results are read or otherwise used in the 
12 computations. In some instances this may result in the same or substantially the 
same allocations as the previous stage, however, where discounting is applied, the 
allocations may not be the same. The extent of the difference may typically depend 
upon the discounting function and the historical prior performance results to which 
the discounting is applied. 

18 Each of these operations is now described in greater detail. For purposes of 

clarity of description, it is assumed that the procedure is applied relative to a single 
zone. If multiple zones are available, the procedure is applied to each of the 
multiple zones one-at-a-time to derive the allocations for that zone. The banner ad 
performance may improve or deteriorate over time so that generally the performance 
of an ad is non-stationary in a statistical sense. There are a number of potential 

24 ' reasons for a banner to have non-stationary behavior in the underlying performance. 

For example, a banner that promises overnight delivery may be quite effective 
shortly before Christmas, but much less effective on the day after Christmas. Even 
absent a particular identifiable event, the performance of ads may change over time. 
For example, Internet visitors may see a given banner too many times and beyond 
some point the banner loses all attraction. As the fraction of visitors who have seen 

30 the ad too many times increases the performance of the banner deteriorates. 
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State Vector Update Procedure 

One embodiment of the inventive procedure optionally but advantageously 
uses a "discounting" of the performance data in computing or updating the State 
Vector. Such discounting may be in the form of a binary step function where data 
older than a certain stage is simply ignored, in the form of a linear or non-linear 
6 weighting function, or according to any other weighting or discounting scheme. In 
one embodiment, the discount is in the form of a geometrical discounting, that is, at 
the end of each stage all data (such as for example, the number of impressions and 
number of clickthroughs) is multiplied by a one-stage discount factor beta, B, where 
beta is a number less than one (B <1). Thus, data that is n stages old at the time 
when the procedure is executed will end up being multiplied by beta raised to the n 

12 power (B n ). In this manner newer performance data is weighted more highly than 

older performance data. While not true in all circumstances, it is generally true that 
newer performance data provides better guidance as to what allocation scheme will 
be successful in the next stage than will older performance data. In any particular 
situation where this generalization does not hold true, different discounting schemes, 
including for example a discounting scheme that weights some segment of 

18 performance data more heavily than newer performance data, may be applied. If no 

discounting is desired, beta may be set to 1 (fl=l). (Empirical and simulation studies 
have shown that a value of the one-stage discount factor (beta) of about 0.9 works 
well for a range of synthetic and real data sets, though values between about 0 and 
about 1 may be used, more typically between about 0.5 and 0.99, more usually 
between about 0.8 and about 1.0, and even more usually between about 0.85 and 

24 0.95 may be used.) 

Let s;(t) denote the total number of discounted cumulative clicks and ni(t) 
denote the discounted cumulative number of impressions for banner i at the end of 
stage t. Let impi(t) denote the total number of impressions and clickj(t) represent 
the total number of clicks and impressions for a particular banner i in stage t. Then 
30 the discounted cumulative clicks Si(f) and the discounted cumulative impressions 
n;(t) are given by the following expressions: 

Si(t) = B Si(t-1) + clicki(t-l), for t > 2 ; and 
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rii(t) = B n ; (t-l) + impi(t-l), for t > 2; 

where sj(l) = 0 and rii(l) = 0. Thus, Si(t) and n;(t) represent two components of the 
State Vector. Discounting reduces the impact of old data and allows the inventive 
6 algorithm to be more responsive to new data that may reflect changes in the 
clickthroughs rates or other performance indicators. 

The above expressions are specialized for an embodiment in which the 
performance metrics are based on numbers of "impressions" and "clickthroughs," 
where for a particular impression the viewer has the binary choice of either clicking- 
12 though or not clicking through for each impression. 

In the set of equations above, the discounting is provided by the factor B. 
The above expressions can be easily generalized to an arbitrary or generalized 
discounting function G{...} which uses the clickthrough impressions from the 
previous stages to derive the discounted cumulative clicks s;(t) and the discounted 
18 cumulative impressions nj(t) using the following expressions: 

Si(t) = G{ clicki(l), clicki(t-2), clicki(t-l)}, for t ^ 2 ; and 

nj(t) = G{ impi(l), . . ., impi(t-2), impi(t-l)}, for t ^ 2 ; 

24 where s;(l) = 0 and n;(l) = 0. In this generalized discounting function G{...}, it is 
noted that none, some, or all, of the clickj(t) and/or impi(t) may actually weighted or 
discounted, though in the preferred embodiment each is discounted. 

In particular functions which give more weight to more recent values of s; 
and ni are of interest. When a different binary response performance indicator than 
30 clickthroughs such as positive responses or "successes" are used, and a more general 
term applied to impressions such as "trials" is used, along with a generalized 
discounting function H {...}, the general expressions for the total number of 



2/14/2010, EAST Version: 



2.4.1.1 



WO 01/48666 



PCT/US00/35408 



-34- 

discounted successes s;(t) and the cumulative number of impressions n ; (t) are given 
by the following expressions: 

s;(t) = H{ successi(l), successi(t-2), success;(t-l)}, for t 2: 2 ; and 

m(t) = H{ triali(l), triali(t-l), triali(t-l)}, for t > 2. 

where Si(l) = 0 and n((l) = 0. 

Thus, Sj(t) and n;(t) represent the State Vector for banner i. As noted above for 
G {...}, none, some, or all, of the successj(t) and/or trial;(t) may actually weighted or 
discounted by the discounting function H {...}, though in the preferred embodiment 
each is discounted. 

In some cases, before the inventive system or method are used for 
optimization, a campaign may already be executing. In this case, information may be 
available regarding the performance of the banners. This performance information 
can be used to initialize s;(l) and n;(l) as follows. A cumulative or discounting 
procedure as described in the previous section can be use to estimate Si(l) and n;(l) 
in the same fashion as Si(t) and nj(t) are calculated, for example, by either using the 
cumulative sum or some form of discounting. Those workers having ordinary skill 
in the art in light of the description provided here will appreciate that there are 
several similar variations of the above basic strategy. 

Even when the campaign is started fresh, a campaign manager may have 
used the same banner in an earlier campaign or may have a prior judgement about 
the performance of the banner. In this case the value of Sj(l) and nj(l) for banner i 
can be set as follows. The manager can provide an estimate of the average 
performance of mean, for the banner i over the previous campaigns. He/she may 
decide that the information from the previous campaign is worth Ni impressions. In 
this case s;(l) can be set to (meani x Ni) and n;(l) can be set to Ni. 
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The setting of s;(l) and n;(l) as described above can be used to incorporate 
managers prior experience or judgement of the banners. Larger values of N; imply 
that the manager is more confident about extrapolating the future performance of the 
banner based on his prior experience or judgement. 

6 Those workers having ordinary skill in the art in light of the description 

provided here will appreciate that there are several similar variations of the above 
basic strategy 

Allocation Computation Procedure 

Having now described several embodiments of the state vector update 
12 procedure, attention is now directed to the allocation computation procedure. At the 
initial stage (when no information has been collected on the banner ads), the 
inventive procedure may allocate the impressions in any one of several possible 
manners. Such initial allocation may be based on objective or subjective criteria. In 
one embodiment, an even or uniform initial allocation of impressions are made from 
among the available banners. That is, if there are a number b banners, then each 
1 8 banner will receive 1/b of the total number of allocated impressions. 

At each of the remaining stages, the inventive procedure uses the available 
performance data (for example, the record of impressions and clickthroughs from 
the earlier stages), allocation information, and other information that may be 
available on the banners to divide the available banners into two classes: (i) a first 

24 group referred to as the contenders, and (ii) a second group referred to as the non- 
contenders. The "contenders" are the banners that are still in contention according 
to some criteria to be declared the best banner. The "non-contenders" are the 
banners that appear out of the running for best banner based on the available 
information. There may also be a (iii) third group of "unavailable banners" if the 
constraint list indicates that some banners cannot be used on the given zone during a 

30 particular stage. Banners that cannot be used for the next stage during which the 
new allocations are to be made would be placed in the third group, as they cannot be 
displayed during the next stage independent of how well they might perform. 
Hence, those ads that are available for the next stage are divided into either the 
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contender or the non-contender group. Any one of these groups may have none, 
one, or a plurality of ads. 

Given this division of the banners into contenders and non-contenders, one 
embodiment of the inventive procedure allocates the impressions among the banners 
as follows. Let y<l be a constant that represents the proportion of impressions that 
will be allocated to the non-contenders at each stage. This y is a constant selected at 
the beginning of the campaign. The value selected is not critical as the inventive 
procedure will reallocate between contenders and non-contenders at subsequent 
stages. The value may also be selected so that impressions are not allocated to non- 
contenders. Let b represent the total number of banners, bi denote the number of 
contenders and b 2 =b-bi the number of non-contenders. Then each contender 
receives the fraction (l-y)/bi of the available impressions, while each non-contender 
receives the fraction y/ba of the available impressions. Using this approach, most of 
the impressions are spread evenly among the contenders and a few impressions are 
used to continue monitoring the non-contenders. As the campaign progresses the 
number of contenders will generally decrease over time since the inventive system 
and method will have more information upon which to judge which banners are 
more successful and which banners are less successful. 

In an allocation task where there are five possible banners {Xi, X2, X3, X4, 
X5}, three of which are contenders {Xi, X2, X3} and two of which are non- 
contenders {X4, X5}, the Allocation Vector may therefore take the form of (0.3, 0.3, 
0.3, 0.05, 0.05) where the contenders each have a 30% proportion and the non- 
contenders each have a 5% proportion of the total. In a situation where there are six 
possible banners but one in constrained by the constraint list for a particular stage, 
that banner would have a zero proportion, for example, allocation vector of the form 
(0.3, 0.3, 0.3, 0.05, 0.05, 0.0). 

Exemplary Pair- Wise Test Procedure to Identify Contenders and Non-Contenders 

In one embodiment, the division of banner ads into contenders and non- 
contenders is based on the following pair-wise test procedure for comparing two 
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banners. When more than two banners are involved in a campaign a pair-wise test is 
used for each possible pair or combination of banners. 

For each pair-wise test, suppose that two banners are given and let p; (for 
i=l, 2) denote the unknown clickthrough probability of the i th banner. Let n; denote 
6 the total number of impressions for the i th banner, s; the total number of successes 
and fi the total number of failures. (So, Si + f; = nj). Finally, define: 

Z = (n 2 Si - nis 2 )/(ni + n 2 ) , and 

V = (n,n 2 (s, + s 2 )(f, + f 2 )/(n, + n 2 ) 3 

12 

It can be shown that Z is approximately normally distributed with variance V and 
mean 0V, where: 

0 = log((p I (l-p 2 ))/(p 2 (l-p 1 ))). 



The pair-wise test attempts to determine or test whether the two unknown 
18 clickthrough rates for the two banners under consideration are equal. If they are 
equal, then 8= 0 and Z is normally distributed with mean 0 and variance V. The 
distance between the observed value of Z and 0 (the mean of its distribution) 
measured in standard deviations is given by Z divided by the square root of V. If 

this statistic (ZA/V) is greater (in absolute value) than some cutoff threshold value a, 

then the procedure concludes that the click-through (or other performance indicia) 
24 rates are significantly different from each other. In one embodiment, the user (such 
as for example the marketing manager) specifies the cutoff threshold value a prior to 
the campaign; however, the value may be modified during the campaign. In one 
embodiment, the value of the cutoff threshold value a is selected to be between 
about V2 and about 1, based on empirical simulation results, but other values may be 
used. 

30 

The pair wise test is applied to all pairs of banners. If a particular banner 
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loses one of these comparisons (that is, if a banner is determined to have a 
significantly lower click-through rate than another banner), then that banner is 
designated as a non-contender. This pair-wise procedure guarantees that at least one 
contender will remain. 

Exemplary Procedure For More Than Two Sets of Alternatives 

The algorithm described above divides the alternatives into two sets: 
contenders and non-contenders. The basic approach can be extended to decompose 
the alternatives into more than two sets. Consider the pair- wise test between 
alternative i and alternative j. Let Compare(iJ) represent the Z(ij)/sqrt(V(ij)) 
statistic where: 

Z(i J) = ( n jSi - niSj)/(ni + nj) , and 

V(i j> (mnj (Si + SjXfi + fj)/(ni + nj) 3 . 

The value of Compare(ij) is positive if the performance of alternative i is better than 
alternative j and is negative if the performance of banner j is better than banner 
alternative i. Define Perf(k) = maxi<j< n Compare(i,k). The algorithm deems an 
alternative k to be a non-contender if Perf(k) is above the cutoff threshold, otherwise 
k is a contender. 

The above basic approach can be extended to decompose the alternatives into more 
than two sets. Under these circumstances, the alternatives are divided into several 
sets based on the corresponding Perf values. The impressions are allocated to 
different alternatives such that the alternatives for sets corresponding to higher 
values of Perf get smaller numbers of impressions. However, the number of 
impressions allocated to all alternatives within a set is equal. In the extreme case, 
the number of sets is equal to the number of alternatives. In this case, each 
alternative gets a different fraction of the total number of impressions (or trials). 
This approach is formalized in the next section. 
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Exemplary Procedure for Dividing Alternatives into Multiple Sets 

The value of Compare(ij) is positive if the performance of alternative i is 
better than alternative j, and the value of Compare(ij) is negative if the performance 
of banner j is better than banner alternative i. Define a performance function 
(Perf(k)) to be the maximum of Compare(i,k) over the range of i from i=l, n; 
6 that is, Perf(k) = maxi<;<n Compare(i,k). The algorithm deems an alternative k to be 
a non-contender if Perf(k) is above the cutoff threshold a, otherwise k is deemed to 
be a contender. 

One embodiment uses the value of Perf(k) to segment the banners into two or 
more sets (r > 2 sets). In this context, larger values of the performance function 
12 Perf(k) as defined here will generally imply poor relative performance of banner 
alternative k. Let there be user defined thresholds a ]} a.2, ... ot(r-i) such that a first 
threshold ao =0, and the remaining thresholds are each greater than oto, that is 0=ao 
< ai < CC2 <...< a( r -i) < a r =oo. The algorithm assigns a banner k to set j, where 
l<j<r, if otfl.i) < Perf(k) < a- 3 . 

18 The impressions are allocated to different alternatives such that the 

alternatives for sets corresponding to higher values of Perf get smaller fraction of 
impressions. However, the number of impressions allocated to all alternatives within 
a set is equal. In the extreme case, the number of sets is equal to the number of 
alternatives. In this case, each alternative gets a different fraction of the total number 
of impressions (or more generally, total number of trials). 

24 

Those workers having ordinary skill in the art in light of the description 
provided here will appreciate that there are several similar variations of the above 
basic strategy that can use the Compare function to segment the banner into multiple 
sets. 

30 One embodiment of the allocation algorithm is as follows. Define constant 

propi, prop 2 , prop 3 , prop r such that propi < prop 2 < prop3 < ... < prop r . Propj 
represents the proportion of total impressions assigned to all the banners belonging 
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to set j, where l<j<r. Let the number of impressions belonging to set j be given by 
numj, where 1 < j < r. (Note that the Sum of numj = total number of banners). Then 
banner belonging to set i is assigned the fraction of total banners = Prop; / {Zi<j< r 
(numj x propj)}. 

6 Unlike the allocation strategy for two sets, the above allocation does not 

guarantee a fixed fraction of impressions to be allocated to relatively under- 
performing sets. However, it guarantees that banners belonging to relatively poor 
performing sets get smaller number of impressions as compared to relatively higher 
performing sets. 

12 Those workers having ordinary skill in the art in light of the description 

provided here will appreciate that there are several similar variations of the above 
basic strategy using the Z and V statistics for the pair-wise comparisons which can 
be used for decomposing the alternatives into two or more sets. 

Procedure For Adding or Deleting Zones or Banners 

18 As mentioned previously, additional operations may be required when zones 

and/or banners are added or deleted. If a zone is deleted, then the number of 
available impressions for this zone is set to 0, and no further banners are allocated at 
this zone (unless the zone is re-introduced later). However, the data collected during 
earlier stages at this zone can be retained. Recall that in some embodiments, the 
earlier collected performance data is aged or discounted so that more recent 

24 performance data is given greater weight in the computations. If a banner is deleted, 
then this information is added to the constraint list so that the banner is no longer 
placed or not placed for a particular zone. Note that a banner may be deleted at one 
or more zones but remain for placement at other zones. After zones or banners have 
been deleted, all inputs to the inventive allocation computation procedure (for the 
remaining zones and banners) are available and allocation for the next stage is 

30 determined. 

If a zone is added, any constraints that will affect the new zone are specified. 
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If a banner is added, the new state vector is initialized (for example, with an empty 
history) and any constraints on the banner are specified. In one embodiment, at the 
next stage, this added banner is treated as though it were as good as the best existing 
banner, and is allocated the same number of impressions as the best banner (for each 
zone). However, this can be easily generalized to allocating more (or less) 
6 impressions than the best banner. Following this initial stage of data collection for 
the new banner, the allocation algorithm is applied as usual. 

Additional E mbodiments 

The above described embodiment (FIG. 3) provides for an external Message 
Server coupled to the database via the Internet through the Message Server 

12 Interaction Service. This approach is advantageous because it allows a loose 
coupling between the optimization system and the Message Server where they may 
be operated by separate business and operational entities, for instance. This is also 
necessary when for various business, security or operational reasons the 
Optimization Service does not have direct access to the database that the Message 
Server uses. Another advantage of this embodiment is that it abstracts out the 

18 functions of each type of message servers so that the optimization system can work 
over different implementations of these servers possibly done by different business 
entities. It also allows the user experience with the overall interface presented by the 
optimization system to be independent of the Message Server, and can continue to 
operate even when the Message Serve configuration is unavailable. However, 
several other alternative configurations of the Message Server, Message Server 

24 Interaction Service, and Database may be used. 

In one of these alternative configurations 200 illustrated in FIG. 6, the 
Message Server 204 is part of an integrated system 202 including the Database 206, 
Optimization engine 208 and the User Interface 210. In this system 200 the 
Optimization Engine 208is integrated right into the Message Server 204 and they 
30 share the same database. The User Interface 210 is integrated directly into the user 
interface of the Message Server - additional user interface elements are added on to 
the Message Server 204 user configuration interface to configure the optimization 
and get the optimization reports. 
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This second alternative configuration is advantageously used when the 
Message Server 204 and the rest of the optimization system can closely be integrated 
without business or operational issues. It has the benefit that the database 
parameters do not need to be copied into another database (other than optimization 
6 that may be done on the message server side for purpose of run-time performance). 

The detriment to integrating the Message Server 204 with the Optimization Engine 
208 is that this may be infeasible because of business and operational necessities. 
Further more, this system will typically lose the flexibility of the being able to 
integrate with multiple message servers that my have different database schemas. 

12 In this embodiment, the Message Server 204 directly enters the message 

performance reports into the database 206, and takes over all of the database activity 
that the MSIS of the embodiment of FIG. 3 would perform. The part of the MSIS 
that talks to the Message Server 204 is no longer needed since the Message Server 
can directly perform the functions internally. The need to get reports, download 
campaign settings, and communicated user and optimization engine settings now 

18 being performed within the integrated system or being eliminated. The functions of 

the optimization system would still exist in a similar fashion. The User Interface 210 
component for interacting with an external user 218 would be altered to add just the 
additional parameters and reports that the optimization system needs or presents. 
Campaign Settings and Optimization Engine Generated Settings are established by 
the OE Engine Service 208 and communicated in a similar manner to the integrated 

24 database component 206. A user input table 216 may be provide within the Message 
server 204. 

In this embodiment, the Job Table data structure is combined with the job 
structure that exists in the Message Server to generate periodic reports. In another 
embodiment, the Job Table remains unchanged, but all the jobs intended for MSIS 
30 in other embodiments are performed directly by the Message Server 204 itself. 

A simpler alternative embodiment of the inventive system 230 illustrated in 
FIG. 7, integrates the optimizing engine (OE) 232 and message server 236 
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performing message allocation and having an internal local data storage 238 or 
database, to an even greater extent. It also places responsibilities for other parts of a 
larger system to other entities while retaining a core that provides optimization and 
allocation. These elements may be combined on or within the same machine or on 
collocated machines and operate with direct communication. The optimizing engine 
6 232 receives optimizing specifications 240 from an external source and generates 
message allocations which it uses to server messages. As in other embodiments 
described here, the messages may be advertisements, emails, and the like, as 
described herein. 

Target Message Destination Types 

12 Although the above described embodiment provides a message server that 

has been described as generally serving a message to a receiver, it is noted that the 
receiver may any one or more of a variety of receiver types. For example, the 
message will frequently be served to a personal computer connected to the world 
wide web via the internet. In other instances, the message will be served via 
interactive television, electronic mail, palm-top information appliances and/or 

18 Personal Data Assistants (PDAs), hand-held wireless devices, or other interactive 
media. 

Advertising 

In this embodiment, the message includes an advertisement for goods or 
services. Typically, the advertisement will be for a commercial product, good, or 
24 service; however, non-commercial advertisements are also anticipated. One type of 
non-commercial advertisement is a so called "public-service" announcement. 

In this embodiment, the Message Server Interaction Service is an 
Advertisement Server Interaction Service, the Message Server is an Ad Server, the 
Message Performance Reports are Ad Performance Reports, the Message Control 
30 Agent is the advertiser, and the Message Campaign is an Ad Campaign. 
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Email 

Email is an important medium for advertising and marketing. Many 
businesses now readily use email to acquire new customers, build brands, advertise 
and promote products, measure consumer satisfaction and manage customer 
relationships. A typical email campaign involves sending emails to each address on 
6 a list of recipients. The list may typically be bought or otherwise acquired from an 
outside firm or collected internally over a period of time. The procedures and 
algorithms developed above can be easily extended for optimizing the responses 
generated by emails. A system and method for improving the performance of 

Emails is described in United States Patent Application Serial No. / 

(Attorney Docket No. A-68760/RMA/LXM), filed _ May 2000 and entitled System 
12 and Method for Optimizing The Performance of Email and Other Message 

Campaigns, which is hereby incorporated by reference. 

The inventive procedures and algorithms may, for example, be extending to 
the email environment by decomposing the email process into multiple stages and 
dividing the measurement process into multiple stages. In each stage a fraction of 

18 total emails to be sent out to each alternative is given by the allocation algorithm 
described above. The major difference in an email application as compared to the 
afore described banner ad application is that there is a non-uniform delay between 
the time an email is sent out and the time a response is received. (For banner ads 
this delay is either non-existent for practical purposes or fairly uniform, for example 
the delay for a banner ad clickthough may be between about 1 second and 60 

24 seconds, well within one stage.) However, one can measure the total number of 
responses received for a given email alternatives by the end of a given stage and the 
total number of emails sent for that alternative as a way of measuring the 
performance of a given alternative. This information can then be used by the 
algorithm to determine and allocate the better alternatives and optimize the total 
number of responses over the entire campaign. The duration of the stage should 

30 advantageously be chosen to ensure that a reasonable fraction of overall responses 
are received by the end of each stage and the number of stages are large enough that 
the algorithm can learn the better performing alternatives in the earlier stages and 
exploit this information in later stages. 
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News 

In yet another embodiment, the message that is being served is a generic 
information item, such as for example, a news headline, a news story, a photograph 
or other graphic, a stock quotation or set of stock quotations, sports news or feature 
6 stories, marketing promotions or coupons, text or images on a web-site, text or 
images on a personal digital assistant, using wireless or other media to connect, is a 
message shown on interactive television. 

Additional Embodiments and Description 

The foregoing description, for purposes of explanation, used specific 
12 nomenclature to provide a thorough understanding of the invention. However, it will 
be apparent to one skilled in the art in light of the description provided that the 
specific details are not required in order to practice the invention. Thus, the 
foregoing descriptions of specific embodiments of the present invention are 
presented for purposes of illustration and description. They are not intended to be 
exhaustive or to limit the invention to the precise forms disclosed, obviously many 
18 modifications and variations are possible in view of the above teachings. 

The embodiments were chosen and described in order to best explain the 
principles of the invention and its practical applications, to thereby enable others 
skilled in the art to best utilize the invention and various embodiments with various 
modifications as are suited to the particular use contemplated. It is intended that the 
24 scope of the invention be defined by the following claims and their equivalents. 
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We claim: 

1. A system for autonomously selecting attributes of a message to improve 
message performance in an interactive measurable medium, said system comprising: 

a database storing message attribute configuration information, including 
6 future message attribute configuration information, and past message performance 
report information; 

an optimization engine receiving said past message performance information 
and making decisions based on said received past message performance information 
and predetermined rules to generate future message attribute configuration 
information; and 

12 a message server proxy that interacts with an external message delivery 

system over a communication link to configure said external system to deliver said 
future message based on said future message attribute configuration information. 

2. The system in claim 1, wherein said predetermined rules include past 
message attribute configuration information associated with said past message 

18 performance information. 

3. The system in claim 2, wherein said predetermined rules further include 
message campaign data and optimization preference data. 

4. The system in claim 1, further comprising a user interface that allows a 
24 message control agent to communicate message campaign data and message 

optimization preference data. 

5. The system in claim 1, wherein said message attribute configuration 
information comprises message control agent configured information and 
optimization engine generated configuration information. 

30 

6. The system in Claim 5, wherein said message control agent comprises an 
external user. 
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7. The system in claim 1, wherein said communication link comprises a 
communication link coupling a network of interconnecting information devices. 

8. The system in claim 1, wherein said communication link comprises a portion 
of the internet. 

6 

9. The system in claim 1, wherein said message server proxy comprises an 
advertising server proxy and said external message delivery system comprises an 
advertising server. 

10. The system in claim 1, wherein said optimization engine includes a 
12 procedure for improving the stage-to-stage performance of a message in a multi- 
stage message campaign in an interactive measurable medium; said procedure 
including steps of: 

(a) reading prior stage message state pertaining to a prior stage in a message 
campaign; said prior stage message state including a cumulative number of trials and 
a cumulative number of successes for a particular (i th ) message at the end of said 

18 prior stage; 

(b) reading message performance results representing message trials and 
message successes from said previous stage based on said prior stage state; 

(c) computing a current message state on the basis of said prior stage 
message state and said message performance results; and 

(d) generating a current message allocation based on said current message 

24 state. 

11. A method for improving the stage-to-stage performance of a message in a 
multi-stage message campaign in an interactive measurable medium; said method 
including steps of: 

(a) reading prior stage message state pertaining to a prior stage in a message 
30 campaign; said prior stage message state including a cumulative number of trials and 
a cumulative number of successes for a particular (i*) message at the end of said 
prior stage; 
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(b) reading message performance results representing message trials and 
message successes from said previous stage based on said prior stage state; 

(c) computing a current message state on the basis of said prior stage 
message state and said message performance results; and 

(d) generating a current message allocation based on said current message 

6 state. 

12. The method in claim 11, wherein said method further comprising step of: 

(e) storing said current message state as prior stage state for a next iteration 
of said method. 

12 13. The method in claim 11, wherein said cumulative number of trials and said 

cumulative number of successes comprise discounted cumulative number of trials 
and discounted cumulative number of successes. 

14. The method in claim 11, wherein said prior stage message state comprises 
reading a state vector for the previous stage in a message campaign. 

18 

15. The method in claim 11, further comprising the step of storing said initial 
state vector in a database. 

16. The method in claim 11, wherein said step of generating a current message 
allocation further includes applying a message allocation constraint. 

24 

17. The method in claim 10, further including the step of storing said current 
message allocation in a database. 

18. The method in claim 12, further including repeating steps (a)-(e) for each 
stage in said message campaign. 

30 

1 9. The method in claim 1 0, wherein said message comprises a web banner ad. 

20. The method in claim 10, wherein said message comprises an email. 
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2 1 . The method in claim 1 0, wherein said method further comprising steps of: 

(e) storing said current message state as prior stage state for a next iteration 
of said method; and 

(f) repeating steps (a)-(e) for each stage in said message campaign; 

6 said cumulative number of trials and said cumulative number of successes 

comprise discounted cumulative number of trials and discounted cumulative number 
of successes; 

said prior stage message state comprises reading a state vector for the 
previous stage in a message campaign; 

said step of generating a current message allocation further includes applying 
12 a message allocation constraint; and 

said message selected from the group of messages consisting of an 
advertisement, an email, and combinations thereof. 

22. A system for autonomously selecting attributes of a message to improve 
message performance in an interactive measurable medium, said system comprising: 

18 a database storing message attribute configuration information, including 

future message attribute configuration information, and past message performance 
report information; 

an optimization engine receiving said past message performance information 
and making decisions based on said received past message performance information 
and predetermined rules to generate future message attribute configuration 
24 information; 

a message delivery system for delivering said future message based on said 
future message attribute configuration information; and 

a message server proxy that interacts with said message delivery system over 
a communication link to configure said message delivery system to deliver said 
future message. 

30 

23. The system in claim 22, wherein said measurable interactive medium is 
selected from the group consisting of the internet, interactive television, email, 
handheld wireless devices, wireless telephones, personal data assistants, and 
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combinations thereof. 

24. The method in claim 22, wherein said message is selected from the group of 
messages consisting or an advertisement, an internet web site banner ad, an email, 
an email advertisement, an email having ad advertisement attachment, a solicitation, 

6 an interactive television message, and combinations thereof. 

25. The system in claim 22, wherein said measurable interactive medium is selected 
from the group consisting of the internet, interactive television, email, handheld 
wireless devices, wireless telephones, personal data assistants, and combinations 
thereof. 

12 

26. The method in claim 22, wherein said message is selected from the group of 
messages consisting or an advertisement, an internet web site banner ad, an email, 
an email advertisement, an email having ad advertisement attachment, a solicitation, 
an interactive television message, and combinations thereof. 

18 27. The system in claim 22, wherein said optimization engine includes a 

procedure for improving the stage-to-stage performance of a message in a multi- 
stage message campaign in an interactive measurable medium; said procedure 
including steps of: 

(a) reading prior stage message state pertaining to a prior stage in a message 
campaign; said prior stage message state including a cumulative number of trials and 

24 a cumulative number of successes for a particular (i th ) message at the end of said 
prior stage; 

(b) reading message performance results representing message trials and 
message successes from said previous stage based on said prior stage state; 

(c) computing a current message state on the basis of said prior stage 
message state and said message performance results; and 

30 (d) generating a current message allocation based on said current message 

state; and 

said state comprises a first state vector component s;(t) for the total number 
of discounted cumulative clicks for message i at the end of stage t, and a second 
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state vector component ni(t) for the cumulative number of impressions for banner i 
at the end of stage t; 
wherein: 

Sj(t) = G{ clicki(l), click;(t-2), clicki(t-l) }, for t > 2 ; 
ni(t) = G { imps( 1 ), . . . , impi(t-2), impi(t- 1 ) } , for t > 2 ; 
where Si(l) = 0 and n^l) = 0; 

impj(t) is the total number of impressions for banner i in stage t; 
clickj(t) represent the total number of clicks for banner i in stage t; and 
G{. . .} is a geometric discounting function. 



28. A method for improving the • stage-to-stage performance of a message in a 
12 multi-stage message campaign in an interactive measurable medium; said method 

including steps of: 

(a) reading prior stage message state pertaining to a prior stage in a message 
campaign; said prior stage message state including a cumulative number of trials and 
a cumulative number of successes for a particular (i £h ) message at the end of said 
prior stage; 

18 (b) reading message performance results representing message trials and 

message successes from said previous stage based on said prior stage state; 

(c) computing a current message state on the basis of said prior stage 
message state and said message performance results; and 

(d) generating a current message allocation based on said current message 

state. 

24 

29. The method in claim 28, wherein said state comprises a first state vector 
component S;(t) for the total number of discounted cumulative clicks for message i at 
the end of stage t, and a second state vector component ni(t) for the cumulative 
number of impressions for banner i at the end of stage t. 



30 30. The method in claim 29, wherein: 

s;(t) = G{ clicki(l), clicki(t-2), click ; (t-l) }, for t > 2 ; and 
n;(t) = G{ impi(l), . . ., imp ; (t-2), impi(t-l) } , for t ^ 2 ; 
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where Si(l) = 0 and nj(l) = 0; and 
where: 

impi(t) is the total number of impressions for banner i in stage t; 
clicki(t) represent the total number of clicks for banner i in stage t; and 
G{. . .} is a discounting function. 

6 

3 1 . The method in claim 30, wherein G { . . . } is a geometric discounting function. 

32. A system for autonomously selecting attributes of a message to improve 
message performance in an interactive measurable medium, said system comprising: 

means for receiving message attribute configuration information and 
12 message performance information; 

an optimization engine receiving said message performance information and 
making decisions based on said received message performance information and 
predetermined rules to generate future message attribute configuration information; 
and 

a message server to deliver messages based on said future message attribute 
18 configuration information. 

33. The system in claim 1, wherein said optimization engine includes a 
procedure for improving the stage-to-stage performance of a message in a multi- 
stage message campaign in an interactive measurable medium; said procedure 
selected from the group of procedures consisting of genetic algorithm based 

24 procedures, neural network based procedures, machine learning based procedures, 
stochastic optimization based procedures, and combinations thereof. 

34. The method in claim 33, further including the step of computing a current 
message allocation and storing said current message allocation in a database. 

30 35. The method in claim 33, wherein said message comprises a web banner ad. 

36. The method in claim 33, wherein said message comprises an email. 
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37. The system in claim 27, wherein said optimization engine includes a 
procedure for improving the stage-to-stage performance of a message in a multi- 
stage message campaign in an interactive measurable medium; said procedure 
selected from the group of procedures consisting of genetic algorithm based 
procedures, neural network based procedures, machine learning based procedures, 

6 stochastic optimization based procedures, and combinations thereof. 

38. The method in claim 37, further including the step of computing a current 
message allocation and storing said current message allocation in a database. 

39. The method in claim 37, wherein said message comprises a web banner ad. 

12 

40. The method in claim 37, wherein said message comprises an email. 
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